0%

(ICCV 2017) Face Synthesis from Visual Attributes via Sketch using Conditional VAEs and GANs

Posted on In , ,

Keyword [Attribute to Face]

Di X, Patel V M. Face synthesis from visual attributes via sketch using conditional vaes and gans[J]. arXiv preprint arXiv:1801.00077, 2017.


1. Overview

In this paper, it proposed Attribute2Sketch2Face



  • A2S. CVAE
  • S2S. GAN
  • S2F. GA

1.1. Motivation

  • stage-wise learning
  • AUDeNet. dense UNet-based
  • Image-to-Image
  • VAE
  • GAN
  • Autoregression

  • CVAE



  • CGAN

    • CycleGAN
    • Wasserstein Distance
    • StackGAN
    • CVAE+GAN


2. Methods



2.1. Attribute-to-Sketch (A2S)



  • encoder q_Φ. encode sketch and encode attribute
  • encoder q_β. encode noise and encode attribute
  • only texture attribute




2.2. Sketch-to-Sketch (S2S)



  • UNet. long skip to preserve low-level features
  • Dense. short skip to maximize information flow
  • D. patch-based
  • only texture attribute


  • VGG of Conv1_2

2.3. Sketch-to-Face (S2F)



  • attribute consist of both color and texture

2.4. Testing




3. Experiments

3.1. Dataset

  • CelebA
  • LFW

  • CUHK

  • use pencil-sketch synthesis method to generate the sketch images from the face images on CelebA and LFW which lack of sketch

  • select attribute of texture and color


3.2. Ablation Study



  • w/o attribute in A2S
  • w/o S2S
  • w/o attribute concat from S2F

3.3. Metric

  • Inception Score
  • Attribute L2. extract attributes from MOON attribute prediction


3.4. Comparison



3.5. Synthesis

  • fix noise z
  • fix attrbute